feat: add filesystem-backed SerDes stage - #648

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Issue Link, if available

Closes #463.

Implements Approach A from ADR-005 and provides the SerDes context requested in #509.

Description

• Extend SerDes with then(SerDesStage), returning the SerDes interface so fluent callers do not depend on the concrete pipeline implementation.
• Define a pipeline as exactly one root SerDes value codec followed by zero or more reversible String → String SerDesStage components. Serialization runs in declaration order and deserialization runs in reverse order.
• Require stages to use self-identifying formats: deserialization reverses recognized valid input, rejects recognized malformed or unsupported input, and passes unrecognized input through unchanged. This lets raw external payloads reach the root value codec without pipeline-specific bypass control flow.
• Keep root SerDes value codecs context-free and pass SerDesContext explicitly only to SerDesStage and nested BinarySerDesStage calls. Serialization-stage context exposes the root input object through originalValue(); deserialization-stage context has a null originalValue(). No SerDes-specific thread-local storage is used.
• Make CloudDurableTestRunner and LocalDurableTestRunner serialize initial invocation input with a separate context-free value codec. By default, they use the persisted SerDes when it is a plain value codec or the root value codec when it is composable; withInputSerDes(...) provides an explicit value-codec override. Persisted pipeline stages are not used to encode initial input.
• Add BinarySerDesStage for reversible byte[] → byte[] transformations and ComposableBinarySerDesStage to expose an ordered binary chain as one top-level string stage.
• Make both binary boundaries customizable through the same StringBinaryCodec interface. The processing-order builder uses startWith(codec), zero or more then(binaryStage) calls, endWith(codec), and build(). The outer stage adds a reserved versioned frame for reliable format recognition.
• Add RetryableSerDesException, RetrySerDesStage, and RetryBinarySerDesStage. The wrappers retry explicitly retryable failures using existing bounded RetryStrategy implementations and pass the same stage context to every attempt.
• Add FileSystemSerDesStage to the core SDK as a reversible SerDesStage for filesystem-backed payload storage. The stage, storage/path configuration, and preview APIs live in the dedicated software.amazon.lambda.durable.serde.filesystem Java package without adding a Maven module.
• Support string payloads, versioned envelopes, ALWAYS/OVERFLOW modes, immutable content-hashed files, attempt-aware operation identities, and pass-through of unrecognized input.
• Publish every offloaded payload to a unique path with one CREATE_NEW write. Publication does not use hard links or renames, so the implementation works on both EFS and S3 Files without overwriting data referenced by an earlier checkpoint.
• Add structured preview parity with the Python and TypeScript SDKs: include-all/exclude-all modes, include/exclude/mask selectors, anywhere/exact-path matching, configurable mask text, a default 4 KB budget, and a standalone preview builder. Custom preview callbacks receive both the preceding stage string and SerDesContext, including originalValue().
• Allow FileSystemSerDesStage to be followed by other stages that transform its inline or file-reference envelope. Filesystem overflow and preview-size checks apply at the filesystem stage boundary; later stages are responsible for their own size expansion.
• Harden filesystem references with producer execution/entity ownership, content-hash validation, filesystem-provider-correct path resolution, path containment, and symlink rejection.
• Carry stage context through durable runtime and persisted-result testing paths, including operation metadata, replay attempts, chained invoke results, executor-backed processing, binary substages, and retry attempts.
• Update the ADR, design, configuration, and SerDes guides for pipeline composition, stage-only context, serialization originalValue, binary stages, retry wrappers, filesystem stages, executor behavior, core SDK packaging, stage format recognition, structured previews, S3 Files compatibility, and independent test-runner input codecs.
• Document a binary-only top-level pipeline as a rejected alternative: it simplifies the type model and can benefit binary-heavy pipelines, but adds full-payload UTF-8 conversion and allocation to the common direct Jackson string path.

Demo/Screenshots

Not applicable. Configuration examples are included in the SerDes guide.

Checklist

• I have filled out every section of the PR template
• I have thoroughly tested this change

Testing

Coverage includes:

• Root SerDes plus SerDesStage pipeline creation, immutable appending, ordering, reverse processing, pass-through of unrecognized input, malformed recognized input, nulls, and failure metadata.
• API validation that only SerDesStage components can be appended after the root value codec and that fluent composition returns the SerDes interface.
• Naming and package audit confirming every named built-in stage implementation ends in Stage and filesystem-specific APIs live under software.amazon.lambda.durable.serde.filesystem.
• Explicit SerDesContext propagation through top-level stages, binary substages, executor-backed calls, and every retry attempt.
• Serialization originalValue() propagation to string stages, binary substages, retry wrappers, and filesystem preview callbacks; deserialization verifies that it is absent.
• Ordered binary serialization and reverse deserialization with custom boundary codecs and versioned format recognition.
• Inline and executor-backed execution, stage context propagation, caching, and executor isolation.
• String and binary retry selection, exhaustion, delay handling, interruption, and context propagation.
• Filesystem stage behavior, stages after filesystem storage, storage modes, envelope recognition and validation, immutable unique publication, providers without hard-link support, ownership, content-hash validation, provider-specific paths, path containment, and symlink rejection.
• Structured preview include/exclude/mask behavior, exact-path and anywhere matching, nested objects and arrays, custom masks, byte budgets, JSON pipeline integration, original-object access, and malformed input.
• Root input/output, step replay and retry attempts, child contexts, callbacks, invoke boundaries, and local/cloud runner behavior.
• Default and custom test-runner input value codecs, persisted-pipeline isolation, composable-input rejection, and filesystem integration without explicit input configuration.

Validated with:

• The full eight-module Maven reactor on Java 17
• Focused filesystem, pipeline, binary-stage, retry, and structured preview tests
• SDK Javadoc generation
• mvn spotless:check
• git diff --check

Cloud integration tests remain opt-in through -Dtest.cloud.enabled=true.

Examples

The SerDes guide, ADR, and design documentation include configuration examples.